Progressive growing of self-organized hierarchical representations for exploration

Designing agent that can autonomously discover and learn a diversity of structures and skills in unknown changing environments is key for lifelong machine learning. A central challenge is how to learn incrementally representations in order to progressively build a map of the discovered structures and re-use it to further explore. To address this challenge, we identify and target several key functionalities. First, we aim to build lasting representations and avoid catastrophic forgetting throughout the exploration process. Secondly we aim to learn a diversity of representations allowing to discover a "diversity of diversity" of structures (and associated skills) in complex high-dimensional environments. Thirdly, we target representations that can structure the agent discoveries in a coarse-to-fine manner. Finally, we target the reuse of such representations to drive exploration toward an "interesting" type of diversity, for instance leveraging human guidance. Current approaches in state representation learning rely generally on monolithic architectures which do not enable all these functionalities. Therefore, we present a novel technique to progressively construct a Hierarchy of Observation Latent Models for Exploration Stratification, called HOLMES. This technique couples the use of a dynamic modular model architecture for representation learning with intrinsically-motivated goal exploration processes (IMGEPs). The paper shows results in the domain of automated discovery of diverse self-organized patterns, considering as testbed the experimental framework from Reinke et al. (2019)

Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems

Self-organization of complex morphological patterns from local interactions is a fascinating phenomenon in many natural and artificial systems. In the artificial world, typical examples of such morphogenetic systems are cellular automata. Yet, their mechanisms are often very hard to grasp and so far scientific discoveries of novel patterns have primarily been relying on manual tuning and ad hoc exploratory search. The problem of automated diversity-driven discovery in these systems was recently introduced [26, 62], highlighting that two key ingredients are autonomous exploration and unsupervised representation learning to describe "relevant" degrees of variations in the patterns. In this paper, we motivate the need for what we call Meta-diversity search, arguing that there is not a unique ground truth interesting diversity as it strongly depends on the final observer and its motives. Using a continuous game-of-life system for experiments, we provide empirical evidences that relying on monolithic architectures for the behavioral embedding design tends to bias the final discoveries (both for hand-defined and unsupervisedly-learned features) which are unlikely to be aligned with the interest of a final end-user. To address these issues, we introduce a novel dynamic and modular architecture that enables unsupervised learning of a hierarchy of diverse representations. Combined with intrinsically motivated goal exploration algorithms, we show that this system forms a discovery assistant that can efficiently adapt its diversity search towards preferences of a user using only a very small amount of user feedback

SBMLtoODEjax: Efficient Simulation and Optimization of Biological Network Models in JAX

Advances in bioengineering and biomedicine demand a deep understanding of the dynamic behavior of biological systems, ranging from protein pathways to complex cellular processes. Biological networks like gene regulatory networks and protein pathways are key drivers of embryogenesis and physiological processes. Comprehending their diverse behaviors is essential for tackling diseases, including cancer, as well as for engineering novel biological constructs. Despite the availability of extensive mathematical models represented in Systems Biology Markup Language (SBML), researchers face significant challenges in exploring the full spectrum of behaviors and optimizing interventions to efficiently shape those behaviors. Existing tools designed for simulation of biological network models are not tailored to facilitate interventions on network dynamics nor to facilitate automated discovery. Leveraging recent developments in machine learning (ML), this paper introduces SBMLtoODEjax, a lightweight library designed to seamlessly integrate SBML models with ML-supported pipelines, powered by JAX. SBMLtoODEjax facilitates the reuse and customization of SBML-based models, harnessing JAX's capabilities for efficient parallel simulations and optimization, with the aim to accelerate research in biological network analysis

Progressive growing of self-organized hierarchical representations for exploration

International audienceDesigning agent that can autonomously discover and learn a diversity of structures and skills in unknown changing environments is key for lifelong machine learning. A central challenge is how to learn incrementally representations in order to progressively build a map of the discovered structures and re-use it to further explore. To address this challenge, we identify and target several key functionalities. First, we aim to build lasting representations and avoid catastrophic forgetting throughout the exploration process. Secondly we aim to learn a diversity of representations allowing to discover a "diversity of diversity" of structures (and associated skills) in complex high-dimensional environments. Thirdly, we target representations that can structure the agent discoveries in a coarse-to-fine manner. Finally, we target the reuse of such representations to drive exploration toward an "interesting" type of diversity, for instance leveraging human guidance. Current approaches in state representation learning rely generally on monolithic architectures which do not enable all these functionalities. Therefore, we present a novel technique to progressively construct a Hierarchy of Observation Latent Models for Exploration Stratification, called HOLMES. This technique couples the use of a dynamic modular model architecture for representation learning with intrinsically-motivated goal exploration processes (IMGEPs). The paper shows results in the domain of automated discovery of diverse self-organized patterns, considering as testbed the experimental framework from Reinke et al. (2019)

Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems

International audienceSelf-organization of complex morphological patterns from local interactions is a fascinating phenomenon in many natural and artificial systems. In the artificial world, typical examples of such morphogenetic systems are cellular automata. Yet, their mechanisms are often very hard to grasp and so far scientific discoveries of novel patterns have primarily been relying on manual tuning and ad hoc exploratory search. The problem of automated diversity-driven discovery in these systems was recently introduced [26, 62], highlighting that two key ingredients are autonomous exploration and unsupervised representation learning to describe "relevant" degrees of variations in the patterns. In this paper, we motivate the need for what we call Meta-diversity search, arguing that there is not a unique ground truth interesting diversity as it strongly depends on the final observer and its motives. Using a continuous game-of-life system for experiments, we provide empirical evidences that relying on monolithic architectures for the behavioral embedding design tends to bias the final discoveries (both for hand-defined and unsupervisedly-learned features) which are unlikely to be aligned with the interest of a final end-user. To address these issues, we introduce a novel dynamic and modular architecture that enables unsupervised learning of a hierarchy of diverse representations. Combined with intrinsically motivated goal exploration algorithms, we show that this system forms a discovery assistant that can efficiently adapt its diversity search towards preferences of a user using only a very small amount of user feedback

Flow-Lenia: Towards open-ended evolution in cellular automata through mass conservation and parameter localization

The design of complex self-organising systems producing life-like phenomena, such as the open-ended evolution of virtual creatures, is one of the main goals of artificial life. Lenia, a family of cellular automata (CA) generalizing Conway's Game of Life to continuous space, time and states, has attracted a lot of attention because of the wide diversity of self-organizing patterns it can generate. Among those, some spatially localized patterns (SLPs) resemble life-like artificial creatures and display complex behaviors. However, those creatures are found in only a small subspace of the Lenia parameter space and are not trivial to discover, necessitating advanced search algorithms. Furthermore, each of these creatures exist only in worlds governed by specific update rules and thus cannot interact in the same one. This paper proposes as mass-conservative extension of Lenia, called Flow Lenia, that solve both of these issues. We present experiments demonstrating its effectiveness in generating SLPs with complex behaviors and show that the update rule parameters can be optimized to generate SLPs showing behaviors of interest. Finally, we show that Flow Lenia enables the integration of the parameters of the CA update rules within the CA dynamics, making them dynamic and localized, allowing for multi-species simulations, with locally coherent update rules that define properties of the emerging creatures, and that can be mixed with neighbouring rules. We argue that this paves the way for the intrinsic evolution of self-organized artificial life forms within continuous CAs

Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems

Self-organisation occurs in many physical, chemical and biological systems, as well as in artificial systems like the Game of Life. Yet, these systems are still full of mysteries and we are far from fully grasping what structures can self-organize, how to represent and classify them, and how to predict their evolution. In this blog post, we present our recent paper which formulates the problem of automated discovery of diverse self-organized patterns in such systems. Using a continuous Game of Life as a testbed, we show how intrinsically-motivated goal exploration processes, initially developed for learning of inverse models in robotics, can efficiently be transposed to this novel application area

IA Curieuse au service de la Science: Découverte Automatisée de Structures Auto-Organisées

Complex systems are very hard to predict and control due to their chaotic dynamics and open-ended outcomes. However, understanding and harnessing the underlying mechanisms of these systems hold great promise for revolutionizing many areas of science. While considerable progress has been made in manipulating and measuring system activity down to the lowest level, there remains a fundamental gap between our knowledge at the micro-level and our ability to control resulting properties on a global scale. Modern machine learning tools offer promising avenues for assisting scientists in navigating the vast space of possible outcomes, especially when aiming for novel or challenging morphological or functional objectives. Nevertheless, current methods tend to constrain and bias the range of events that AI can measure and attempt to influence. This thesis aims to transpose and advance recent computational models of intrinsically motivated learning and exploration with the goal of designing more open-ended forms of AI discovery assistants for assisting scientists in mapping the outcome space of self-organizing systems. To that end, several key ingredients are introduced to efficiently shape the discovery process. These include the use of unsupervised learning for representations, meta-diversity search, curriculum learning, and external human guidance, whether environment-based or preference-based. We discuss how these components, when implemented in practice, can help address challenging problems in science. These challenges encompass the search for interesting patterns in continuous models of cellular automata, the investigation of the origins of sensorimotor agency, the exploration of gene regulatory networks behavioral capabilities, and the design of innovative forms of cellular collectives for applications in AI and biology.Les systèmes complexes sont très difficiles à prédire et à contrôler en raison de leur dynamique chaotique et de leurs vastes espaces de sortie. Cependant, comprendre et exploiter les mécanismes sous-jacents de ces systèmes offre de grandes promesses pour révolutionner de nombreux domaines scientifiques. Bien que des progrès considérables aient été réalisés dans la manipulation et la mesure de l'activité des systèmes jusqu'au niveau microscopique voire nanoscopique, un fossé fondamental persiste entre nos connaissances à l'échelle microscopique et notre capacité à contrôler les propriétés résultantes à l'échelle globale. Les outils modernes d'apprentissage automatique offrent des perspectives prometteuses pour aider les scientifiques à naviguer dans l'espace complexe des sorties du système, en particulier lorsqu'il s'agit d'atteindre de nouveaux buts morphologiques ou fonctionnels difficiles. Néanmoins, les méthodes actuelles ont tendance à restreindre et à biaiser l'étendue des événements que l'IA peut mesurer et tenter d'influencer. Cette thèse vise à appliquer et à développer les modèles computationnels récents d'apprentissage et d'exploration intrinsèquement motivés dans le but de concevoir des assistants de découverte IA pour aider les scientifiques à cartographier les résultats potentiels des systèmes auto-organisés. Pour atteindre cet objectif, plusieurs éléments clés sont introduits pour façonner efficacement le processus de découverte. Cela comprend l'utilisation de l'apprentissage non supervisé de représentations, la recherche de méta-diversité, l'apprentissage par curriculum, et l'intégration de guidage humain dans la boucle (par l'introduction de contraintes environnementales ou de préférences). Nous discutons de la manière dont ces composants, lorsqu'ils sont mis en pratique, peuvent contribuer à résoudre des problèmes scientifiques complexes. Cela comprend la recherche de motifs intéressants dans des modèles continus d'automates cellulaires, l'investigation des origines de l'agence sensorimotrice, l'exploration des capacités comportementales des réseaux de régulation génétique et la conception de formes innovantes de collectifs cellulaires pour des applications en IA et en biologie

Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems

Source code and videos athttps://automated-discovery.github.io/International audienceIn many complex dynamical systems, artificial or natural, one can observe self-organization of patterns emerging from local rules. Cellular automata, like the Game of Life (GOL), have been widely used as abstract models enabling the study of various aspects of self-organization and morphogenesis, such as the emergence of spatially localized patterns. However, findings of self-organized patterns in such models have so far relied on manual tuning of parameters and initial states, and on the human eye to identify "interesting" patterns. In this paper, we formulate the problem of automated discovery of diverse self-organized patterns in such high-dimensional complex dynamical systems, as well as a framework for experimentation and evaluation. Using a continuous GOL as a testbed, we show that recent intrinsically-motivated machine learning algorithms (POP-IMGEPs), initially developed for learning of inverse models in robotics, can be transposed and used in this novel application area. These algorithms combine intrinsically-motivated goal exploration and unsupervised learning of goal space representations. Goal space representations describe the "interesting" features of patterns for which diverse variations should be discovered. In particular, we compare various approaches to define and learn goal space representations from the perspective of discovering diverse spatially localized patterns. Moreover, we introduce an extension of a state-of-the-art POP-IMGEP algorithm which incrementally learns a goal representation using a deep auto-encoder, and the use of CPPN primitives for generating initialization parameters. We show that it is more efficient than several baselines and equally efficient as a system pre-trained on a handmade database of patterns identified by human experts